Characterization and Control of Hysteretic Dynamics Using Online Reinforcement Learning

Hysteretic dynamical systems are challenging to control due to their hard nonlinearity and difficulty in modeling. One type of systemwith hysteretic dynamics that is gaining use in aerospace systems is the shape-memory alloy-based actuator. These actuators provide aircraft and spacecraft systems with the ability to achieve component-level or vehicle-level geometry or shape changes. Characterization of thematerial dynamics and properties of these actuators is usually accomplished with empirical testing of physical specimens, in which the hysteresis dynamics are often abstracted to very simplified models or ignored entirely. Machine learning techniques have the potential to learn hysteretic dynamics, but they routinely encounter difficulties that make them unsuitable. This paper proposes and develops a reinforcement learning-based approach that directly learns an input–output mapping characterization of hysteretic dynamics, which is then used as a control policy. A hyperbolic tangent-based model is used to develop a simulationof a shape-memory alloy,which is then validated experimentally using the Sarsaalgorithm.The simulation model produces the temperature-versus-strain behavior and characterizes both themajor andminorhysteresis loops. The learning results produce a near-optimal control policy for modulating a shape-memory alloy wire to a specified length. Results presented in the paper show that casting the shape-memory alloy control problem as a reinforcement learning problem shows promise for characterizing and controlling shape-memory alloy hysteresis behavior.